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Focus on the future, Informed with the past 
• Questions for NASA Human Research Program 

- How do we best inform decision makers and mission 
planners on what is needed to allow humans to visit 
Moon, Mars, and other celestial bodies? 

- How do we optimize medical needs for astronauts with 
limited Mass, Volume and Power? 
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Integrated Medical Model (IMM) 

“The most important questions of life are indeed, for the most part, really only problems 



of probability. ” ~ Pierre Simon Laplace 



Potential Medical 
Condition 


Evaluate 
with IMM 



Clinical Evidence 


Impact 

Health 


Likelihood of occurrence, 
probable severity of 
occurrence, and 
optimization of treatment 
and resources. 


• Integrate best available evidence to quantify the probability and consequences 
of medical risks 


• Identify medical resources and inform operational decisions which optimize 
health and mission success 


“Data hungry” process 
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Houston, we have a problem 







Forecasting rare medical events 
confounded by space travel 

- Insufficient data 

• Few or no occurrences 

- No clear correlation to terrestrial 
analog 

- Multiple influencing conditions 

• Mission details 

• Vehicle limitations (environment) 

• Physiological adaptations 

• Interaction of the contributing factors 

How do we verify and validate? 




- When we are data “starved” 
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Its Really About Model Credibility ! 

Achieving a high level of belief or trust in the model 





N AS A-STD-7 009 

- Standard for Models 
and Simulations (M&S) 

M&S Development 

- Verification 

- Validation 

M&S Operations 

- Input Pedigree 

- Results Uncertainty 

- Results Robustness 

Supporting Evidence 

- Use History 

- M&S Management 

- People Qualifications 



People Qualifications^^" Validation 


M&S Management 


Input Pedigree 


Use History^-. ^Results Uncertainty 

Results Robustness 


^“Sufficiency Thresholds 
Factor Score 

Human research program: 

a) Scoring subject to SME review 

b) Results used to balance project 
derived evidence 
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Facts are Stubborn Things 

Credibility Levels of Evidence 




Climb the 
Ladder 


Level 

Verification 

Validation 

Input Pedigree 

Results 

Uncertainty 

Results 

Robustness 

Use History 

M&S 

Management 

People 

Qualifications 

4 

Numerical 
errors small 
for all 
important 
features. 

Results agree 
with real- 
world data. 

Input data agree 
with real -world 
data. 

Non- 

detennmistic 
& numerical 
analysis. 

Sensitivity 
known for 
most 

parameters. 

key* 

sensitivities 

identified 

De facto 
standard. 

Continual 

process 

improvement. 

Extensive 
experience m 
and use of 
recommended 
practices for this 
particular M&S. 

3 

Formal 

numerical 

error 

estimation. 

Results agree 
with 

experimental 
data for 
problems of 
interest. 

Input data agree 
with 

experimental 
data for 
problems of 
interest. 

Non- 

determmistic 

analysis. 

Sensitivity 
known for 
many 
parameters 

Previous 
predictions 
were later 
validated by 
mission data. 

Predictable 

process. 

Advanced 
degree or 
extensive M&S 
experience, and 
recommended 
practice 
knowledge. 

2 

Unit and 
regression 
testing of 
key features. 

Results agree 
with 

experimental 
data or other 
M&S on unit 
problems. 

Input data 
traceable to 
formal 

documentation. 

Deterministic 
analysis or 
expert 
opinion. 

Sensitivity 
known for a 
few 

parameters. 

Used before 
for critical 
decisions. 

Established 

process. 

Formal M&S 
training and 
experience, and 
recommended 
practice training. 

1 

Conceptual 

and 

mathematical 

models 

verified. 

Conceptual 

and 

mathematical 
models agree 
with simple 
referents. 

Input data 
traceable to 
informal 
documentation. 

Qualitative 

estimates. 

Qualitative 

estimates. 

Passes 
simple tests. 

Managed 

process. 

Engineering or 
science degree. 

0 

Insufficient 

evidence. 

Insufficient 

evidence. 

Insufficient 

evidence. 

Insufficient 

evidence. 

Insufficient 

evidence. 

Insufficient 

evidence. 

Insufficient 

evidence. 

Insufficient 

evidence 


M&S Development 

M&S Operations 

Supporting Evidence 
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Dose Likelihood 





Sleep Disruption- Medical Intervention Forecasting 

(SDMIF) Tool 

Predict the sleep inducing 
medication use during a space 
mission 



r MONTE CARLOllMULAfiON 



- Resulting from circadian rhythm 
disruption, insomnia, and 
environmental disruptions 


Probability Distribution of Taking a Sleep Aid on a Simulated ISS Increment 
(180 days, nominal sleep/wake opportunities, 5000 iterations) 
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SDMIF: Verification and Validation 



• Limited space flight data most likely used in the 
development of the model 

• Verification exercises 

- Fixed and Extreme value testing of all PDFs 

- Estimates of numerical error 

• Validation of individual components 

- Validate conceptual structure by SME review 

• Schedulers verify mission schedule component 

• Medical Ops validate the diagnosis component 

• Validation of Module Performance 

- Face validation as V&V tool 

• Turing or Schruben tests with operational and flight 
medical experts 



- Direct comparison to observed incidence 

• Historical testing - Select data used for validation 

• Prospective validation - Future missions observations 
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Model Data, Robustness and Uncertainty 







Model Data 

- Desired: Assure data are appropriate for the 
intended model use 

- Achieved: Highest quality of the data 
correlated to the scenario 

Model Robustness 

- Desired: percentage of the contribution of an 
independent variable to the variation of the 
outcome 

- Achieved : Rank order correlation sensitivity 
analysis 

Model Uncertainty 

- Desired: magnitude and confidence of 
estimate 

- Achieved: quantified based on non- 
deterministic analysis 
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Use History, Management and Qualifications 

• Use History 

- Desired: Model use extended to address 
similar questions 

- Achieved: 

• Similar outputs to other tools in limited scenarios 

• When Model is used to inform real world 
decisions 

• Management: Continuous Improvement 

- Desired and Achieved: Document all activity, 
management processes and decisions 
affecting code development, input changes, 
and V&V efforts. 

• Qualifications 

- Desired: Staff can interpret and use the results 

- Achieved: Development staff maintains 
expertise levels required to develop, maintain, 
update and operate the model environment. 
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Implementation in Global IMM 



• Implementation requires that modules 

- Undergo subject matter review 

• SDMIF - Conceptual model, Input data, 
validation process reviewed by National 
Space Biomedical Research Institute 

- Undergo integration team review 

• IMM team review of module performance 
and evidence 

- Credibility MUST be at least that of the rest 
of the IMM 

- New IMM model undergoes V&V and 
credibility assessment 

• All evidence meticulously documented 

- “We can lick gravity, but sometimes the 
paperwork is overwhelming.” ~Wernher 
von Braun 



ii 
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Let’s Review 



• Models related to medical events used to inform Space 
Flight Operations and Planning 

- Must exhibit a high level of Credibility for the intended use 

- Must have their Credibility assessed over multiple factors 

• Chief factor is an acceptable V&V process 

- Complicated by limited in-flight data 

- Relies on systematic use of SME’s and Face Validation to balance 
limits in direct comparative evidence 
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